Method and system for managing data access requests utilizing storage meta data processing

ABSTRACT

A method and system are provided for managing requests to access data in a computer environment. In one aspect of the present invention, a request manager receives a request associated with meta data corresponding to data that is maintained separately from the meta data. In another aspect, the request manager informs a data object manager of an anticipated request that will be received by the data object manager to enable it to prepare for the anticipated request. The data object manager commences preparing for the anticipated request in response to being informed of the anticipated request to facilitate a reduction in data access time. In one example of a computing environment utilizing one or aspects of the invention, a storage subsystem comprising a data object manager prepares for the anticipated request by pre-fetching data blocks from storage media into a cache.

TECHNICAL FIELD

This invention relates, in general, to the management of requests toaccess data in communications environments, and more particularly, toinforming a data object manager of an anticipated request to access thedata from storage media based on a received request associated with metadata which corresponds to the data.

BACKGROUND OF THE INVENTION

Storage subsystems generally consist of a number of disk drives that canbe aggregated and made to appear as virtual disk drives to one or moreclient computers. To improve performance, storage subsystems usuallydeploy a cache which is used to hold frequently accessed disk blocks.The choice of which disk blocks to cache can have a significant impacton overall system performance. Some storage subsystems attempt toanticipate which disk blocks may be required by client computers byexamining historical patterns of access to disk blocks. The nature ofsuch cache management algorithms is predictive.

Although there are techniques today for the management of requests toaccess data in communications environments, these techniques can cause astorage subsystem to load data into its cache that is not accessedwithin the expected time because of their predictive nature. Thus, thereis still a need for further techniques to facilitate the management ofrequests to access data in computer environments.

SUMMARY OF THE INVENTION

The shortcomings of the prior art are overcome and additional advantagesare provided through the provision of a method of managing requests. Inone aspect, a manager receives a request associated with meta datacorresponding to data maintained separately from the meta data. Inanother aspect of the present invention, the manager informs anothermanager of an anticipated request that will be received by the anothermanager to enable it to prepare for the anticipated request.

Systems and computer program products corresponding to theabove-summarized methods are also described and claimed herein.

Additional features and advantages are realized through the techniquesof the present invention. Other embodiments and aspects of the inventionare described in detail herein and are considered a part of the claimedinvention.

BRIEF DESCRIPTION OF THE DRAWINGS

The subject matter which is regarded as the invention is particularlypointed out and distinctly claimed in the claims at the conclusion ofthe specification. The foregoing and other objects, features, andadvantages of the invention are apparent from the following detaileddescription taken in conjunction with the accompanying drawings inwhich:

FIG. 1 illustrates a flowchart of one embodiment of a technique formanaging requests associated with data in a computer environment, inaccordance with an aspect of the present invention;

FIG. 2 illustrates a flowchart of another embodiment of a technique formanaging requests associated with data in a computer environment, inaccordance with an aspect of the present invention;

FIG. 3 illustrates one embodiment of a technique for managing requestsfor access to data in an environment in which data and meta dataassociated with the data are stored separately, in accordance with anaspect of the present invention;

FIG. 4 illustrates an example of an environment in which a technique formanaging requests for access to data is utilized, in accordance with anaspect of the present invention;

FIG. 5 illustrates another example of an environment in which atechnique for managing requests for access to data is utilized, inaccordance with an aspect of the present invention; and

FIG. 6 illustrates a third example of an environment in which atechnique for managing requests for access to data is utilized, inaccordance with an aspect of the present invention.

BEST MODE FOR CARRYING OUT THE INVENTION

In one aspect of the present invention, a manager receives a requestassociated with meta data. The manager informs another manager of ananticipated request to be received the another manager to enable theanother manager to prepare for the anticipated request.

A technique for managing requests associated with data in a computerenvironment in accordance with an aspect of the present invention isdescribed below with reference to request management flowchart 60illustrated in FIG. 1. First, step 61 comprises a request managerreceiving a request associated with meta data. Then, in step 62, therequest manager informs a data object manager of a change in a dataobject's meta data. The data object manager makes a data objectmanagement decision in step 63 and, if necessary, acts to implement thedata object management decision in step 64.

Further aspects of a technique for managing requests associated withdata objects in a computer environment in accordance with the presentinvention are described below with reference to flowchart 50 illustratedin FIG. 2. First, step 51 comprises a request manager receiving a usagerequest. Then, if the communications unit is granted permission to use adata object as determined in step 52, a request manager sends arequest-management message to a data object manager and responds,substantially simultaneously, to a communications unit with ausage-request response in steps 53 and 55, respectively. Subsequent tosteps 53 and 55, respectively, the data object manager prepares for ananticipated request in step 54, and data-usage communications aretransmitted between the communications unit and data object manager instep 56. In one example, the step of transmitting data-usagecommunications includes transmitting by a communications unit requestsfor data blocks and transmitting by a data object manager datacomprising the requested data blocks of the data object. Alternatively,if the communications unit is not granted permission to use a dataobject, step 57 comprises the request manager responding to acommunications unit with a usage-request response.

One example of a communications environment in which a technique formanaging requests associated with data objects is utilized in accordancewith an aspect of the present invention is described below withreference to FIG. 3. In an environment in which the meta data associatedwith data objects may be stored separately from the data objects, arequest manager 10 receives usage requests from a communications unit 20via request management network 12. Request manager 10 transmits arequest-management message to a data object manager 30 via a privatenetwork 14 and responds to communications unit 20 with a usage-requestresponse via request management network 12. If communications unit 20 isgranted permission to use a data object 40, the communications tosupport use of data object 40 are transmitted via data network 16between communications unit 20 and data object manager 30.

Generally, both meta data and user data are associated with data objectsin a computer communications environment. User data is the informationthat has meaning to a user or to a program that may process that data.Examples of user data are the contents of a Freelance Graphics®presentation, or employee information stored within a relationaldatabase. Meta data is information about user data. Examples of metadata associated with data objects include the identity of clientcomputers that have access permission, data object type, names of filesassociated with a set of disk blocks, the length of a file, the list ofblocks that constitute a file, information about user accesspermissions, and the date and time a file has been created or updated.Data objects comprise data. Data object types include data files,checkpoint files, file systems, logical volumes, and journaled filesystem (JFS) logical volume logs.

Features facilitated by use of the technique in the computer environmentillustrated in FIG. 3 include: improved security with respect to accessto data objects, regulation of the speed of access to data objects,arbitration of access priorities to data objects, and increased speed ofaccess to data objects.

An emerging class of storage environments separates the storage of userdata and meta data and provides separate networks over which the userdata and meta data traverse. An example of such a storage environment isIBM's Storage Tank™ file system wherein a Storage Tank™ client (acomputer) accesses user data from a storage subsystem (over a storagearea network (SAN) using block transfer protocols) and accesses metadata from a centralized Storage Tank™ meta data controller (overEthernet using TCP/IP protocols). The separation of user data and metadata can be either logical or physical. Storage subsystems, whichgenerally comprise a number of disk drives that can be aggregated andmade to appear as virtual disk drives to one or more client computers,usually deploy a cache, which is used to hold frequently accessed diskblocks, to improve input-output performance.

One or more aspects of the present invention take advantage of the factthat where user data and meta data are separated, the processing of metadata in conjunction with file access provides additional informationwhich can be used to inform a storage subsystem of future input/output(I/O) access requests. This information can be utilized by a storagesubsystem to facilitate the management of its internal caches.

An example of the management of the contents of a cache in a datastorage subsystem which utilizes information obtained by processing filemeta data in accordance with an aspect of the present invention isdescribed as follows with reference to FIG. 4. When a client computer210 wants to read and update some disk blocks associated with a filelocated in a storage subsystem 230, client computer 210 must be grantedan exclusive lock on the associated disk blocks. Client computer 210initiates a transaction to meta data controller 220, requesting thelock. This lock request indicates that client computer 210 intends toperform I/O operations on a certain range of disk blocks in the future.If meta data controller 220 can grant the requested lock, meta datacontroller 220 passes a “hint” to storage subsystem 230, which storesthose blocks, indicating that the storage subsystem can expect toreceive an I/O request from a client computer for a particular range ofdisk blocks. Meta data controller (MDC) 220 communicates this “hint” tostorage subsystem 230 via private network 222. Essentially concurrently,meta data controller 220 grants the lock by signaling to client computer210 via meta data network 212. In this exemplary embodiment, MDC 220 isan example of a request manager, and storage subsystem controller 236 ofstorage subsystem 230 is an example of a data object manager.

If storage subsystem 230 determines that the requested disk blocks arenot in cache 232, it pre-fetches the requested blocks from storage disks234 into cache 232. After receiving the requested lock, client computer210 initiates an I/O operation with storage subsystem 230 via datanetwork 214 to access at least some of the disk blocks on which a lockwas received. When the client-initiated I/O request is received bystorage subsystem 230, the storage subsystem may have the requested diskblocks in its cache already as a result of pre-fetching. If not, storagesubsystem 230 has already commenced the necessary physical I/O to loadthe requested blocks into cache 232 as a result of previously receivinga hint from meta data controller 220. When the requested disk blocks areavailable in cache 232, they are sent to client computer 210 from cache232 via data network 214. The result of storage subsystem 230 initiatingdisk input/output in order to store disk blocks that are subject to afuture access request by a client computer in cache 232 in advance ofreceiving a request from client computer 210 is that data access latencyis reduced.

The method of the present invention is also utilized in the operation ofthe example illustrated in FIG. 4 when a client computer writes diskblocks to the storage subsystem. If client computer 210 sends atransaction to MDC 220, indicating that the client computer has closed afile and has finished writing blocks to the storage subsystem, meta datacontroller 220 communicates this information to storage subsystem 230via private network 222. Storage subsystem 230 determines whether tofree the storage locations in cache 232 in which the disk blockscomprising the closed file are stored based on this file-closed messageand possibly “hints” received regarding other future data accessrequests. Freeing storage locations in cache 232 permits storage ofother disk blocks in cache for expedited access.

Another example of managing the contents of a cache in a data storagesubsystem which utilizes information obtained by processing file metadata in accordance with an aspect of the present invention relates to acomputer writing a large file which is not likely to be read. An exampleof such a file is a checkpoint/restart file, created by a long-runningcomputational job, or a database log file. These files are typicallyused to recover the state of a computational workload after a computercrash. Since computer crashes are very rare, checkpoint/restart filesare typically written regularly, but rarely read. Knowledge of thisinformation can be used to inform a storage subsystem not to cache acheckpoint/restart file once it has been written to disk.

This example is described further with reference to the exemplaryenvironment of FIG. 4. When computer 210 wishes to write acheckpoint/restart file, it requests write permission from meta datacontroller 220 via the meta data network 212 and informs meta datacontroller 220 that the file should not be actively cached.Alternatively, meta data controller 220 could automatically recognizethat the file is of a particular type (such as a checkpoint/restartfile). Meta data controller 220 grants permission to computer 210 towrite the file, providing a list of blocks where the file should bewritten, and simultaneously, via private network 222, informs storagecontroller 236 of storage subsystem 230 that computer 210 is about towrite a large file that should not be actively cached in storagesubsystem cache 232.

When computer 210 writes the file via data network 214 to storagesubsystem 230, storage subsystem controller 236 decides how much ofcache 232 to allocate to storing all or part of the large file and, asquickly as possible, writes the contents of the large file to storagedisks 234 within the storage subsystem. As soon as the contents (orpartial contents) of the file are written to storage disks 234 of thestorage subsystem, the associated file data within the cache 232 can bediscarded immediately, since it is highly unlikely that this file willneed to be read again. Thus, utilization of the cache based on knowledgeof the type of data being stored, which is gained through processing thefile's meta data, facilitates optimization of the use of the cacheresource.

An example related to the previous example involves a computer reading acheckpoint/restart file described above to recover the state of acomputational workload after the computer has crashed. Knowledge thatthis type of file is rarely read can be used to inform the storagesubsystem to not cache the file when it is being read from disk. Itshould be noted that the management of the contents of a cache in a datastorage subsystem described below with respect to this example appliesto reading any large file which is only likely to be accessedinfrequently.

With reference to FIG. 4, computer 210 intends to read acheckpoint/restart file so computer 210 requests permission from metadata controller 220 via meta data network 212 to read the file andinforms meta data controller 220 that the file should not be activelycached. Alternatively, meta data controller 220 automatically recognizesthat the file is of a particular type (such as a checkpoint/restartfile), rather than having to be informed by computer 210. Meta datacontroller 220 grants permission to computer 210 to read the file,providing a list of blocks where the file is located, andsimultaneously, via the private network 222, informs storage subsystemcontroller 236 of storage subsystem 230 that computer 210 is about toread a large file that should not be actively cached in the storagesubsystem cache 232. When computer 210 reads the file via data network214 from storage subsystem 230, storage subsystem controller 236 decideshow much of cache 232 to allocate to reading the file from storage disks234. As soon as the contents (or partial contents) of the file aretransmitted to computer 210, the associated file data within the cache232 can be discarded immediately, since it is highly unlikely that thisfile will need to be read again. Thus, utilization of the cache based onthe type of data being accessed, which is determined by processing thefile's meta data, frees cache resources to facilitate faster access toother files.

Another example of an environment using a technique for managingrequests for access to data objects to effectuate management of thecontents of a cache in a data storage subsystem by utilizing informationobtained from processing file meta data in accordance with an aspect ofthe present invention is described below with reference to FIG. 5. Inthis example, a number of computers (310, 311, and 312) are members of adatabase cluster. Each of computers 310, 311, and 312 hosts an instanceof the cluster's database. When a computer 310 wants to read and updatesome disk blocks associated with a database table located in a storagesubsystem 330, computer 310 must be granted an exclusive lock on theassociated disk blocks. Computer 310 requests a lock on those diskblocks from database lock manager 320 by sending a request on externalnetwork 314. If database lock manager 320 can grant the requested lock,database lock manager 320 grants the lock via a message sent to computer310 on external network 314 and essentially simultaneously sends amessage to storage subsystem 330, indicating that the storage subsystemcan expect to receive an I/O request from a client computer for aparticular range of disk blocks. Database lock manager 320 communicatesthis message to storage subsystem 330 via private network 322.Analogously to the previous example, database lock manager 320 is anexample of a request manager, and storage subsystem 330 comprises a dataobject manager.

If storage subsystem 330 determines that the disk blocks for which alock was granted are not in cache 332, storage subsystem 330 initiatesan I/O operation to storage disks 334. Computer 310 initiates an I/Ooperation with storage subsystem 330 since it has been granted a lock onthe requested disk blocks. When the I/O request initiated by computer310 is received by storage subsystem 330 via data network 316, thestorage subsystem may have already pre-fetched the requested disk blocksand stored them in cache 332. Even if all requested disk blocks have notyet been loaded in the cache, the storage subsystem has initiated thephysical I/O from storage disks 334 prior to receiving the request fromcomputer 310. As a result, the latency in providing the requested diskblocks from cache 332 is less than it would be without pre-fetchingprompted by the “hint” from the database lock manager.

In another example of a computer environment embodying the presentinvention, which is described with reference to FIG. 6, a centralizedstorage meta data controller 434 is co-hosted with a storage subsystem420. In this environment, one or more computers are connected to thedata storage subsystem via a data network. By way of example, as shownin FIG. 6, a computer 410 exchanges data with a storage subsystem 420via data network 412. Storage subsystem 420 comprises a server 430connected to a cache 322 and storage disks 424. Server 430 compriseslogical partitions 431 and 432.

In this example, the functions of meta data controller 434 and storagesubsystem controller 433 are executed by software running in logicalpartitions (LPARs) 432 and 431, respectively, of server 430. Usingvirtual input/output bus 436 between the meta data controller LPAR 432and the storage subsystem controller LPAR 431, “hints” regardinganticipated future I/O requests directed to storage subsystem 420 arepassed at very high speed and low latency from meta data controller 434to storage subsystem controller 433. The benefit of pre-fetching diskblocks into the storage subsystem cache is enhanced by the use ofhigh-speed, low-latency communications between the meta data controllerand storage subsystem controller. In this example, meta data controller434 and storage subsystem controller 433 are examples of a requestmanager and a data object manager, respectively.

The present invention can be included in an article of manufacture(e.g., one or more computer program products) having, for instance,computer usable media. The media has therein, for instance, computerreadable program code means or logic (e.g., instructions, code,commands, etc.) to provide and facilitate the capabilities of thepresent invention. The article of manufacture can be included as a partof a computer system or sold separately.

Additionally, at least one program storage device readable by a machineembodying at least one program of instructions executable by the machineto perform the capabilities of the present invention can be provided.

The flow diagrams depicted herein are just examples. There may be manyvariations to these diagrams or the steps (or operations) describedtherein without departing from the spirit of the invention. Forinstance, the steps may be performed in a differing order, or steps maybe added, deleted or modified. All of these variations are considered apart of the claimed invention.

Although preferred embodiments have been depicted and described indetail herein, it will be apparent to those skilled in the relevant artthat various modifications, additions, substitutions and the like can bemade without departing from the spirit of the invention and these aretherefore considered to be within the scope of the invention as definedin the following claims.

1. A method of managing requests in a communications environment, saidmethod comprising: receiving by a manager a request associated with metadata, said meta data corresponding to data maintained separately fromthe meta data; and informing, by the manager, another manager of ananticipated request to be received by the another manager to enable theanother manager to prepare for the anticipated request.
 2. The method ofclaim 1, further comprising preparing by the another manager for theanticipated request, said preparing responsive to said informing.
 3. Themethod of claim 2, wherein said preparing comprises managing contents ofa cache in a data storage subsystem.
 4. The method of claim 2, whereinsaid preparing comprises managing a user's or a client computer's accessto the data.
 5. The method of claim 2, further comprising: sending, bythe manager, a reply to a communications unit in response to the requestsubstantially simultaneously with said informing; and receiving, by theanother manager, the anticipated request, wherein said preparing beginsbefore the receiving by the another manager.
 6. The method of claim 3,wherein said managing contents comprises pre-fetching one or more datablocks from one or more storage media of the data storage subsystemwhereby the data blocks are stored in the cache, the data blockscomprising at least some of the data.
 7. The method of claim 3, whereinsaid managing contents comprises releasing storage locations of thecache whereby the storage locations become available for storing otherdata, the storage locations storing data blocks comprising at least someof the data.
 8. A request management system for a communicationsenvironment, said system comprising: means for receiving by a manager arequest associated with meta data, said meta data corresponding to datamaintained separately from the meta data; and means for informing, bythe manager, another manager of an anticipated request to be received bythe another manager to enable the another manager to prepare for theanticipated request.
 9. The system of claim 8, further comprising meansfor preparing by the another manager for the anticipated request, saidmeans for preparing responsive to said means for informing.
 10. Thesystem of claim 9, wherein said means for preparing comprises means formanaging contents of a cache in a data storage subsystem.
 11. The systemof claim 9, wherein said means for preparing comprises means formanaging a user's or a client computer's access to the data.
 12. Thesystem of claim 9, further comprising: means for sending, by themanager, a reply to a communications unit in response to the requestsubstantially simultaneously with informing the another manager of theanticipated request to be received; and means for receiving, by theanother manager, the anticipated request, wherein said means forpreparing begins prepare for the anticipated request before the meansfor receiving receives the anticipated request.
 13. The system of claim10, wherein said means for managing contents comprises means forpre-fetching one or more data blocks from one or more storage media ofthe data storage subsystem whereby the data blocks are stored in thecache, the data blocks comprising at least some of the data.
 14. Thesystem of claim 10, wherein said means for managing contents comprisesmeans for releasing storage locations of the cache whereby the storagelocations become available for storing other data, the storage locationsstoring data blocks comprising at least some of the data.
 15. At leastone program storage device readable by a machine embodying at least oneprogram of instructions executable by the machine to perform a method ofmanaging requests in a communications environment, said methodcomprising: receiving by a manager a request associated with meta data,said meta data corresponding to data maintained separately from the metadata; and informing, by the manager, another manager of an anticipatedrequest to be received by the another manager to enable the anothermanager to prepare for the anticipated request.
 16. The at least oneprogram storage device of claim 15, wherein said method furthercomprises preparing by the another manager for the anticipated request,said preparing responsive to said informing.
 17. The at least oneprogram storage device of claim 16, wherein said preparing comprisesmanaging contents of a cache in a data storage subsystem.
 18. The atleast one program storage device of claim 16, wherein said preparingcomprises managing a user's or a client computer's access to the data.19. The at least one program storage device of claim 16, wherein saidmethod further comprises: sending, by the manager, a reply to acommunications unit in response to the request substantiallysimultaneously with said informing; and receiving, by the anothermanager, the anticipated request, wherein said preparing begins beforethe receiving by the another manager.
 20. The at least one programstorage device of claim 17, wherein said managing contents comprisespre-fetching one or more data blocks from one or more storage media ofthe data storage subsystem whereby the data blocks are stored in thecache, the data blocks comprising at least some of the data.